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In the early days of the International Space Station (ISS) program, and as the organization 
structure was being internationally agreed upon and documented, one of the principal 
tenets of the science program was to allow customer-friendly operations. One important 
aspect of this was to allow payload developers and principle investigators the flexibility to 
operate their experiments from either their home sites or distributed telescience centers. 
This telescience concept was developed such that investigators had several options for ISS 
utilization support. They could operate from their home site, the closest telescience center, 
or use the payload operations facilities at the Marshall Space Flight Center in Huntsville, 
Alabama. The Payload Operations Integration Center (POIC) processes and structures 
were put into place to allow these different options to its customers, while at the same time 
maintain its centralized authority over NASA payload operations and integration. For a 
long duration space program with many scientists, researchers, and universities expected to 
participate, it was imperative that the program structure be in place to successfully facilitate 
this concept of telescience support. From a payload control center perspective, payload 
science operations require two major elements in order to make telescience successful within 
the scope of the ISS program. The first element is decentralized control which allows the 
remote participants the freedom and flexibility to operate their payloads within their scope 
of authority. The second element is a strong ground infrastructure, which includes voice 
communications, video, telemetry, and commanding between the POIC and the payload 
remote site. Both of these elements are important to telescience success, and both must be 
balanced by the ISS program’s documented requirements for POIC to maintain its 
authority as an integration and control center. This paper describes both elements of 
distributed payload operations and discusses the benefits and drawbacks. 
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perspective, payload science operations require two major elements in order to make 
telescience successful within the scope of the ISS program. The first element is decentralized 
control which allows the remote participants the freedom and flexibility to operate their 
payloads within their scope of authority. The second element is a strong ground 
infrastructure, which includes voice communications, video, telemetry, and commanding 
between the POIC and the payload remote site. Both of these elements are important to 
telescience success, and both must be balanced by the ISS program’s documented 
requirements for POIC to maintain its authority as an integration and control center. This 
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I. History of Science Operations 

F ROM the earliest days of space flight, science has been an integral part of the mission. Data collected for 
monitoring vehicle systems and astronauts has always been needed to verify the novel challenges of space 
exploration. Today with the Space Shuttle and International Space Station (ISS) we tend to think of science as the 
specialized experiments conducted solely for the scientific result. The amazing paradigm shift is a testament to the 
advances of modern space science operations and the development of a true space laboratory which is accessible by 
users all over the world. 

When the space program started, the communication and computing systems were primitive, by today’s 
standards, which forced ground controllers, including scientists, to perform their operations from centralized control 
centers. It required dedicated communication lines to allow control centers to interact. The transfer of commands 
and data could only be performed at slow rates. This forced the early space controllers and scientists to come 
together at centralized locations to support their space flight equipment. Central control centers were established by 
NASA at the major centers. NASA established and paid for the necessary communications lines and command and 
data management equipment. Scientists were required to travel to the control centers to operate their payloads. 
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1980 


1985 


Spacelab 1-3: Flight Controllers and Scientists support from the 
JSC Payload Operations Control Center. 


1990 


Spacelab Astro- 1: Spacelab Flight Controllers and Scientists support from the 
MSFC Payload Operations Control Center in Huntsville. 


1995 


All Spacelab Missions: Limited capability for 
Scientists to support from remote locations. 


2000 

2005 

2010 


ISS 5A.1: Payload Flight Controllers support from the MSFC Payload 
Operations Integration Center in Huntsville. Scientists support from 
remote locations worldwide. 


ISS IE: ESA Flight Controllers 
support from Columbus Control 
Center near Munich. 


ISS 1/JA: JAXA Flight Controllers 
support from JAXA Control Center 
near T sukuba. 


ISS ULF-7: End of Shuttle flights. ISS support 
continues from worldwide locations. 



Figure 1. Historical Timeline of Science Operations 


A. Skylab and Early Spacelab 

Skylab was the first attempt to create a dedicated laboratory facility in space, but its main focus was to extend 
man’s spaceflight capability. Longer duration effects on astronauts and vehicle systems were the main focus of 
Skylab. Naturally, the primary location for this activity was the Mission Control Center in Houston (MCC-H). 
Teams of subject matter experts would travel to Houston to monitor data collected on controlled activities with the 
crew and vehicle during these longer duration mission. Data was often collected and archived to be analyzed over 
next few months and years by the experts in their own facilities. Any experiment modification based on data would 
have to be scheduled for a new mission, perhaps years in the future. Initial Spacelab missions shortened this turn 
around since the Space Shuttle could launch a new laboratory into space, rather than just the individual test items. 
Operations, however, remained much the same as Skylab for the early Spacelab missions. 
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For the Spacelab missions from 1985 to 1990, all Marshall Space Flight Center (MSFC) flight controllers and 
scientists travelled to the Johnson Space Center (JSC) for operations. This involved a team of over 100 people 
travelling several times for training simulations and again for the flight itself. The lives of those travelling were 
disrupted, sometimes for weeks at a time. Integrated training and coordination opportunities were limited due to the 
logistical nightmare involved with hundreds of travelers. 

B. Later Spacelab 

Later, Spacelab added two major components to the space laboratory: a dedicated science control center and a 
dedicated laboratory with modular facilities. Scientists worked with the Payloads Operations Control Center 
(POCC) to design and install specialized analysis equipment in the control center. This maximized science return by 
doing preliminary analysis of the data and allowing the opportunity to modify science activities to improve the 
results. Training and preparation for the mission with the Payload Crew Training Complex (PCTC) allowed 
developers to optimize the experiment configuration for crew operations and train specialists to operate their 
facilities. With the modular facility design, scientist began to develop experiments unrelated to crew or vehicle 
survival. Crystal and plant growth experiments are some examples of science based on the microgravity 
environment, rather than crew or vehicle survival. Scientists began to see Spacelab as a laboratory. It still, 
however, took considerable effort to build both the ground and space portions of the laboratory. 

Beginning with the Astro- 1 mission in 1991, the POCC and Payload Training Control Center (PTCC) were 
developed that allowed Spacelab flight controllers and scientists to operate from MSFC in a dedicated Science 
Operations Area (SOA). While those capabilities allowed MSFC-based flight controllers to train and operate at 
MSFC, most scientists and some non-MSFC-based flight controllers were still required to travel to MSFC. 

Scientists operating their experiments from a central control center, instead of their home sites, face many 
challenges. They must segment their team to those that can travel and those that support from home. They may not 
have their engineering support equipment available to troubleshoot unexpected results or malfunctions. Their 
normal methods of coordination and communication get disrupted. Additionally, the ground hardware used at the 
central control center may not be in the same configuration as their home site. 

II. Beginnings of Remote Operations 

Remote operations began to develop as Spacelab progressed. The first few missions were operated solely from 
the Johnson Space Center. The logistics of transporting teams and equipment to a centralized facility was a costly 
and enormous task. STS-61A Spacelab D1 (Deutschland- 1) was controlled by operators in Oberpfaffenhofen, 
Germany and significantly challenged the need for a central location. In a centralized control center, data was 
collected, stored, and copied to portable media. Eventually the data was shipped back to the scientist’s home 
location, but the need or desire for more timely feedback began to push the use of developing communication 
technology. 

Operations were beginning to move to other control centers, like MSFC, which managed the Spacelab program. 
Initially, the advantage of sending segments of data back to the home facility meant fewer members needed to travel 
to support operations. Many of the early Spacelab missions, especially observatory missions, like STS-35 ASTRO- 
1, would send experts to ensure the right data was captured and recorded. As technology increased, voice 
conversations turned into emails, and file transfers allowed data confirmation and number crunching. More team 
members could review data electronically sent to them. The advent of email, file transfer protocol (FTP), and the 
internet, allowed for the transfer of nonoperational and noncritical data. Safety of the payloads and the crew still 
restricted critical data to dedicated transmission lines and dedicated control centers. 

By STS-56 Atmospheric Laboratory for Applications and Science-2 (ATLAS-2), coordination with international 
partners had increased significantly. Cost and effort to route and deroute lines for individual facilities was not a cost 
effective solution. Sharing critical data with remote users was a highly desired option. Passwords and accounts 
gave some control of data access. Internet encryption and virtual private networks (VPNs) began to allow users to 
securely access telemetry from remote locations. The security required for command capability would take all of 
these developments put together. 

ISS distributed operations did not develop overnight. MSFC was charged with development of the Chandra X- 
ray Telescope and control center. The ground system was developed and then deployed in Cambridge, MA. Many 
of the skills and techniques need for distributed operations were learned by remotely deploying a control center. 
When developers began to extend development to the ISS control center, they were able to incorporate and scale the 
technology learned from Chandra. ISS distributed operations are a result of several program and partners 
incorporating new technology to make an overwhelming task easier. 
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A. Why Distributed Operations? 

ISS operations brought on the concept of distributed operations and remote users. With 24 x 7 operations, more 
science planned, and a more geographically dispersed set of users (worldwide), the need for remote operations 
became evident. NASA began to work more and more with international partners, expanding the logistical 
problems of moving teams of people and equipment to a centralized location. Fortunately, during the same period, 
communications and the internet were also growing rapidly. Payload operations were able to take advantage of the 
change in communications to allow users to stay at their home facility and collect data, providing flexibility to 
scientists around the world. Being able to access data around the world greatly reduced the cost of research and 
allowed more researchers to be involved in the investigations. 

B. HOSC Capabilities to Early Remote Users 

Early remote operations mainly concentrated on connecting the control team with the experts who, for cost or 
schedule reasons, could not be present at the control center. Most of the early efforts allowed the experts to be 
available for questions as operations deviated from the expected, as it almost always does. Phone and voice loops 
were extended to limited locations. These early efforts provided flexibility to the scientists while enabling 
operations. 

Spacelab D1 was an effort to include the European Space Agency (ESA), who had a vested interest in the 
science, in the actual control of the operations. Data, voice, and control were extended to ESA for the dedicated 
mission. Missions like STS-35 ASTRO-1, which was basically a flying observatory, also pushed the need for 
interaction between the astronomers and the control team, especially with the Instrument Pointing System (IPS) 
alignment issues experienced on ASTRO-1. Images and star data were transmitted to external users. Transferring 
files via FTP and other methods was invaluable. Microgravity missions, such as United States Microgravity 
Laboratory- 1 (USML-1) with the Extended Duration Orb iter (EDO), meant there was more time on orbit to react to 
the unexpected results and modify the experiments to capture better science. Universities all over the world needed 
to coordinate the results. Email, a staple of universities, was widely used to share information. STS-56 
Atmospheric Laboratory for Applications and Science-2 (ATLAS-2), with its earth sensing experiments, piqued the 
interest of users who were widely scattered across the world. 

Methods had to be employed which were common to all interested parties. Some of the early Spacelab mission 
required users to use specific systems (voice keysets, computers, and software) to communicate. The Chandra 
telescope and ISS purposely used hardware and systems which were more ubiquitous to ease some of the burden of 
external users. There was a transition from Spacelab mission support environment for minicomputer architecture to 
client server model. As technology advanced and users began to be more familiar with the personal computer (PC) 
platform and the World Wide Web took off, a transition to a PC -based architecture was needed. This made for 
lower cost deployable systems at remote sites. ISS Telescience Resource Kit (TReK) software to be discussed later 
was a result of this cumulative experience. 

C. Technology Growth 

One of the major factors in the success of remote operations has been the development of technology to support 
it. In the early days of space science, data had to be recorded to mainframe tapes in select groupings and either 
carried or shipped to the destination. The progress in early Spacelab was to buy dedicated lines to transmit vital 
information. Technology developments in protocols for data transmission greatly aided the amount and speed by 
which data could be transmitted. 

Establishing Telescience Support Centers (TSCs) with dedicated lines and secure protocols was critical to 
involving other research centers, universities and international partners. The advent of the internet and development 
of secure protocols allowed telescience to reach around the world. The connections once dedicated between two 
control centers could now be connected through existing electronic paths paid for and established by the users. 

Distributing the required resources and the control of the data distribution to the end user allowed the Payload 
Operations and Integration Center (POIC) to concentrate on being a hub for data transmission, not just an end point. 
The developers at the Huntsville Operations Support Center (HOSC) took off-the-shelf protocols and pieced them 
together to form a secure and reliable network with the end user. User downloaded software to interface with the 
control center and a VPN network allowed multiple types of data to be transmitted securely. Better transmission 
lines for the general public continue to increase the ability of users to collect and share data at a higher rate. 

III. Current Distributed Operations 
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The POIC is an ISS facility that manages on-orbit 1SS payloads and payload support systems in coordination 
with the MCC-H, the International Partner (IP) Payload Control Centers (PCCs), TSCs, and payload-unique 
facilities, as shown in Fig. 2. It is responsible for planning, integration, and operation of ISS on-orbit payloads. It 
is physically located within the HOSC at MSFC. The POIC provides command, telemetry, database, planning, 
information management systems, voice services, etc. for local, remote, and a globally distributed ISS payload user 
community. 



International Space 
Station (ISS) 


Globally 
Distributed 
Remote 
Payload 
Users & 
Facilities 


SCIENCE OPERATIONS 

POIC Inteqration/Services For Remote Users 
Payload Command Uplink Gateway. 
Downlink Vehicle/Payload T elemetry Distribution 
Voice Comm Control. Data T ransfer Services. 
Planning Services. Information Systems, etc 


Figure 2. ISS Distributed Paylod Operations 


A. HOSC Capability Upgrades 

Some of the HOSC capability upgrades included the ability to send commands and receive telemetry from the 
ISS payload. Currently in the POIC, controllers use the Enhanced HOSC System (EHS) UNIX server-based EHS 
Personal Computer (EPC) client architecture to access EHS payload operations tools and services within the HOSC. 
This same capability can be utilized by scientists and payload developers around the world. EPC software runs on 
the Windows XP platform. EPC users can share products with other users and run the products locally on the PC. 
Such products include: Figure 3 shows the current architecture of data flow to and from the HOSC, IPs, TSCs, 
remote sites, MCC-H, and the ISS. 

• Telemetry Displays 

• User-developed Computations 

• Command Groups 

• Automated Scripts 

EPC users also have access to an extensive scripting language that can be used for automated telemetry 
acquisition, command updates, and command uplinks. EPC allows users to: 

• Receive and display telemetry data on a user-defined display 

• Perform computations on the received telemetry values 

• Continuously monitor specific telemetry parameters to detect anomalies 

• Update and uplink commands to the spacecraft 

• Track and verify command uplinks 
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One of the tools that used for remote operations is TReK. TReK is a suite of PC -based software applications 
used by scientists and engineers to monitor and control payloads on-board the 1SS. The PC running the TReK 
software can be located anywhere in the world. This provides a way for scientists and engineers to monitor and 
control experiments located in space from their offices and laboratories at home. 
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Figure 3. Remote Operations Architecture 


Another tool that is used is a Voice Over Internet Protocol (VOIP) system called Internet Voice Distribution 
System (IVoDS). It was developed to support ISS audio communications. IVoDS allows Internet-based 
participants to talk/listen on one conference or channel, while monitoring up to 24 others. This multi-conferencing 
system links researchers, POIC operations controllers, and the ISS crew together to support collaboration during 
Space Station experiment operations and planning. It was originally created for remote users but has since been 
incorporated as the primary voice system at the POIC. 

IV. Benefits of Distributed Operations 

There are many benefits to distributed operations. Distributed operations allow flexibility in scheduling science 
operations, since support is not dependent on travel. With distributed operations, science teams can better manage 
their personnel and have access to all of their team. It also eliminates travel costs and travel time overhead. A big 
benefit is the ability to provide long-term support since the scientists operates from their home base (e.g. multi- 
week, multi-month, or longer studies). Additionally, scientists have local access to their support personnel and 
equipment to respond to unexpected results or malfunctions. 

One benefit that has grown from the initial distributed operations is the capability to provide an ISS Backup 
Control Center (BCC) capability at MSFC. With the experiences from Hurricanes Lili, Katrina, and Rita, it was 
apparent there was a need to establish a mission control alternative to sole reliance on the MCC-H at JSC. In 
response, ISS Program Management and the operations teams at both JSC and MSFC implemented an ISS BCC at 
MSFC. The BCC provides continuous mission control for system and payloads operations aboard the ISS, even in 
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the event of catastrophic loss of the MCC-H capability using the client server PC capability (EPC) and the IVoDS 
voice system. 

With the remote operations capability offered, the JSC flight control team could conduct mission operations 
from Huntsville, either remotely or by physically relocating to MSFC, in the event of any contingency. With a 
direct communications path to White Sands Complex (WSC), MSFC is the only NASA Center, besides JSC, to 
command the ISS. 


V. Drawbacks 

There are always some drawbacks to any new concept. With all the new payloads and updates that need to be 
made prior to a new crew on ISS (an “Increment” for planning and operations purposes) there are several technology 
issues. One is an increase in the complexity of transitions and software and database testing. Every Shuttle flight 
and Increment requires a new database or software transition with new payload and science information. It is a 
concerted effort to keep up with versions of software and operating systems among remote centers and the POIC. 
This is mitigated by the process called Certification of Flight Readiness (COFR). A team of people manages the 
COFR process to make sure that each system, software, and user are flight-ready. 

Other issues came with the use of commercialized carriers for communication lines around the world. If a major 
communication line was lost (e.g. during the Japanese earthquake in March 2011), ISS operations could lose a major 
capability such as voice, video, commanding, or telemetry between control centers and the ISS crew. This risk is 
mitigated by the use of redundant circuits; however there is always a slight possibility that both redundant 
capabilities could be lost. 

One other major downfall is the loss of face-to-face communication benefits and synergies. Sometimes the need 
to be able to physically talk or relay a point is needed. Many times in conversations or electronic transfer of 
information the point is lost. This is something that the POIC works diligently to overcome by scheduling face-to- 
face meetings to iron out potential issues before they are experienced in real-time operations. 

VI. Conclusion 

The remote operations concept for ISS was developed to allow payload developers and principal investigators 
the flexibility to operate their experiments from either their home sites or distributed telescience centers. The POIC 
maintains its centralized authority over NASA payload operations and integration, while allowing the remote 
participants the freedom and flexibility to operate their payloads within their scope of authority. The ground 
infrastructure, including voice communications, video, telemetry, and commanding, which has evolved from the 
Skylab era, supports this arrangement. 


Acronyms 

BCC - Backup Control Center 

COFR - Certification of Flight Readiness 

EHS - Enhanced HOSC System 

EPC - Enhanced HOSC System (EHS) Personal Computer 

ESA - European Space Agency 

FTP - File Transfer Protocol 

HOSC - Huntsville Operations Support Center 

IP - International Partner 

ISS - International Space Station 

IVoDS - Internet Voice Distribution System 

JSC - Johnson Space Center 

MCC-H - Mission Control Center - Houston 

MSFC - Marshall Space Flight Center 

PC - Personal Computer 

PCC - Payload Control Center 

PCTC - Payload Crew Training Complex 

POCC - Payload Operations Control Center 

POIC - Payload Operations Integration Center 

PTCC - Payload Training Control Center 

SOA - Science Operations Area 

STS - Space Transportation System 
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TReK - Telescience Resource Kit 
TSC - Telescience Support Center 
WSC - White Sands complex 
VOIP - Voice Over Internet Protocol 
VPN - Virtual Private Networks 
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